Image processing apparatus, image processing method, image processing program recording medium, color adjustment method, color adjustment device, and color adjustment control program recording medium

ABSTRACT

In conventional image processing techniques, although a certain kind of image adjustment provides a desired effect on an image, it may cause an undesired side effect thereon, resulting in an unsatisfactory result of adjustment.  
     On a computer serving as the nucleus of image processing, a region applicable to image processing is specified at a step, an object pixel is moved for judging whether or not it belongs to the specified region at the subsequent steps, and then a specified image processing operation is carried out if the object pixel is judged to belong to the specified region. Therefore, adjustment of image data in a certain region does not have an adverse effect on image data in other regions, making it possible to realize satisfactory adjustment in an entire image with ease.  
     Further, on the computer, chromaticity “x-y” of each pixel is calculated at a step, and statistical calculation is performed at the subsequent steps if a value of chromaticity thus calculated is in a possible chromaticity range. After completion of statistical calculation on all the pixels, an average value is determined at the subsequent step, and a degree of each color adjustment is calculated while taking account of the number of pixels of interest at the subsequent step. In this manner, accurate statistical calculation is performed on color pixels to be adjusted independently of brightness, and a degree of each color adjustment is regulated by taking account of the number of pixels of interest, thereby making it possible to carry out optimum color adjustment processing without giving an adverse effect on colors of surrounding pixels.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an image processing apparatus,an image processing method and an image processing program recordingmedium for inputting digital photographic image data containingdot-matrix pixels and converting each pixel of the image data accordingto predetermined correspondence relationship information, and it is alsoconcerned with a color adjustment method, a color adjustment device anda color adjustment control program recording medium for carrying outoptimum color adjustment.

[0003] 2. Description of Related Art

[0004] There are a variety of conventional image processing techniquesfor processing digital photographic image data, more specifically suchimage processing techniques for contrast enhancement, chromaticitycorrection, and brightness correction, for example. Each pixel of imagedata is converted according to predetermined correspondence relationshipinformation in these conventional image processing techniques. In anexample of chromaticity correction, a color conversion table is preparedbeforehand, and when original image data is input, the color conversiontable is referenced to produce output image data. Thus, in flesh colorcorrection, for example, a flesh color part of an image is made vivid onoutput.

[0005] In conventional image processing techniques mentioned above,however, when correction of certain chromaticity is made, an entireimage may be affected by the correction, causing an unsatisfactoryresult of image adjustment. For instance, if it is attempted to make apallid complexion of a person's image look ruddy, an entire image maybecome reddish. Namely, although a certain kind of image adjustmentprovides a desired effect on an image, it may cause an undesired sideeffect thereon.

SUMMARY OF THE INVENTION

[0006] It is therefore an object of the present invention to provide animage processing apparatus which is capable of minimizing an adverseside effect in adjustment of an image data.

[0007] In accomplishing this object of the present invention andaccording to one aspect thereof, there is provided an image processingapparatus for inputting photographic image data containing dot-matrixpixels and converting each pixel of the image data according topredetermined correspondence relationship information, which comprisesthe correspondence relationship holding unit which holds a plurality ofcorrespondence relationship information of the image data in combinationwith applicable object information thereof, the correspondencerelationship judging unit which judges each correspondence relationshipfor converting each pixel of the image data according to thecorrespondence relationship and applicable object information which areheld by the correspondence relationship holding unit, and the image dataconverting unit which converts each pixel of the image data byreferencing each correspondence relationship judged by thecorrespondence relationship judging unit from the correspondencerelationship holding unit.

[0008] Where an image contains dot-matrix pixels and each pixel isindicated in image data representation, each pixel of image data isconverted according to predetermined correspondence relationshipinformation in the above-mentioned arrangement of the present invention.In this image processing, a plurality of correspondence relationshipinformation for image data conversion and applicable object informationare held in combination. According to these information, a judgment isformed on each correspondence relationship for conversion of each pixelof image data, and based on result of judgment, conversion of each pixelof image data is carried out.

[0009] More specifically, there are provided plural correspondencerelationships for image data conversion, and it is judged whichcorrespondence relationship is to be applied to each pixel. Then, imagedata conversion is performed by applying a proper correspondencerelationship to each pixel. For example, a correspondence relationshipfor making green color of tree leaves more vivid is applied to greencolor pixels indicating tree leaves, and a correspondence relationshipfor making a complexion ruddy is applied to flesh color pixels. In thismanner, each correspondence relationship is flexibly applied asrequired.

[0010] In accordance with the present invention, since correspondencerelationship information for image data conversion can be changed foreach pixel, it is possible to provide an image processing method whichallows execution of desired image processing without giving an adverseeffect to unspecified parts of an image.

[0011] It will be obvious to those skilled in the art that the imageprocessing method in which plural correspondence relationshipinformation can be changed for each pixel is practicable not only in asubstantial apparatus, but also in a system on its method. From thispoint of view, the present invention provides an image processingapparatus of inputting photographic image data containing dot-matrixpixels and converting each pixel of the image data according topredetermined correspondence relationship information, in which aplurality of correspondence relationship information for conversion ofthe image data and applicable object information thereof are held incombination, each correspondence relationship for converting each pixelof the image data is judged according to the correspondence relationshipand applicable object information thus held, and each pixel of the imagedata is converted according to result of judgment.

[0012] The present invention for realizing image processing as mentionedabove may be practiced in a variety of forms, i.e., it may be embodiedin hardware, software or combination thereof in various arrangementsthat are modifiable as required. As an example of practicing the presentinvention, it may be embodied in image processing software. In thiscase, according to the present invention, there is provided a softwarestorage medium for recording the image processing software. From thisviewpoint, the present invention provides a storage medium for recordingan image processing program for inputting photographic image datacontaining dot-matrix pixels on a computer and converting each pixel ofthe image data according to predetermined correspondence relationshipinformation, the image processing program comprising the steps of;holding a plurality of correspondence relationship information forconversion of the image data in combination with applicable objectinformation thereof, forming judgment on each correspondencerelationship for converting each pixel of the image data according tothe correspondence relationship and applicable object information thusheld, and converting each pixel of the image data according to result ofjudgment thus formed. In this case, the same functionalities as in theforegoing description can be provided.

[0013] The storage medium for recording the image processing program maybe a magnetic storage medium, a magneto-optical storage medium or anysoftware storage medium to be developed in the future. Obviously, thepresent invention embodied in image processing software may take such areplicated form as a primary replicated product, secondary replicatedproduct, etc. In addition, the image processing software according tothe present invention may be supplied through use of a communicationline or in a form of contents written in a semiconductor chip.

[0014] Still more, there may be provided such an arrangement that someparts of the present invention are embodied in software while the otherparts thereof are embodied in hardware. In a modified embodiment of thepresent invention, some parts thereof may be formed as software recordedon a storage medium to be read into hardware as required.

[0015] In image processing according to the present invention,photographic image data is particularly suitable as object data to beprocessed. However, it is not necessarily required to use 100%photographic image data as object data, and semi-synthetic image data orcomposite-photographic image data produced from plural photographicimages may be used as object data.

[0016] As stated above, in accordance with the present invention, thereare provided a plurality of correspondence relationships in combinationwith applicable object information thereof. Each correspondencerelationship is used to perform specific conversion of input image data,and for each correspondence relationship, there is provided applicableobject information thereof. It is to be understood that such applicableobject information may take a variety of forms.

[0017] Another object of the present invention is to provide an imageprocessing method of using applicable object information.

[0018] According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, each correspondence relationship applicable toeach part of image is held along with information on each partapplicable to each correspondence relationship, and wherein, in thecorrespondence relationship judgment means, a part of imagecorresponding to each pixel is detected and each correspondencerelationship for conversion is judged through comparing the detectedpart of image with part information for each correspondencerelationship.

[0019] In the above-mentioned arrangement of the present invention, aunique correspondence relationship is applicable to each part of image,and there is provided part information applicable to each correspondencerelationship. Therefore, a part of image corresponding to each pixel isdetected, and the detected part of image is compared with partinformation for each correspondence relationship. Thus, if a match isfound, image data conversion is carried out using relevantcorrespondence relationship information.

[0020] More specifically, a unique correspondence relationship isprovided for each part, of image and it is applied to image data to beconverted. For instance, a correspondence relationship for sky-bluecolor is applied to an upper part of image, while performing noconversion on a lower part of image. In this case, a part of image isnot limited in its size and shape, and it may take any shape such as arectangular shape, round shape, free shape, etc. Furthermore, it is notnecessarily required to specify a part of image in a particular range. Apart of image may be specified with respect to a certain center point.In this case, a degree of adjustment in image conversion may bedecreased gradually as being apart from the center point. Obviously, acertain correspondence relationship may be applied to a plurality ofparts of image.

[0021] As stated above, in accordance with the present invention, sinceeach correspondence relationship is changed with each part of image, itis possible to carry out image processing with a particular locationspecified, thereby permitting easy operation.

[0022] Another object of the present invention is to provide an imageprocessing method of specifying an applicable object.

[0023] According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, a plurality of correspondence relationshipsare held along with information on chromaticity applicable to eachcorrespondence relationship, and wherein, in the correspondencerelationship judgment means, a value of chromaticity of each pixel isdetected and each correspondence relationship for conversion is judgedthrough comparing the detected value of chromaticity with information onchromaticity for each correspondence relationship.

[0024] In the above-mentioned arrangement of the present invention, aunique correspondence relationship is applicable to each level ofchromaticity. Therefore, a value of chromaticity of each pixel isdetected, and the detected value of chromaticity is compared withinformation on chromaticity for each correspondence relationship injudgment on each correspondence relationship for conversion. In thiscase, chromaticity is used for the purpose of judgment on applicabilityof each correspondence relationship, i.e., it is not necessarilyrequired to adjust a level of chromaticity.

[0025] In accordance with the present invention, since an object can beselected using chromaticity, its embodiment is relatively easy even ifpixels to be processed are located at a plurality of parts of image.

[0026] In chromaticity judgment, input image data may be used intactly,or a color space scheme is changed before judgment so that input imagedata is converted into data easy for judgment on chromaticity. Stillmore, it is not necessarily required to use a narrow definition ofchromaticity. Chromaticity judgment may be made in a wider definition byusing a level of any predetermined recognizable characteristic of imagedata.

[0027] It is possible to adopt various units for setting correspondencerelationship information for image data conversion.

[0028] Another object of the present invention is to provide an imageprocessing apparatus of using a color conversion table.

[0029] According to the present invention, there is provided an imageprocessing method wherein, in the step of holding the correspondencerelationship information, a color conversion table is used for storinginformation on correspondence relationship between pre-convertedoriginal image data and post-converted image data.

[0030] In the above-mentioned arrangement of the present invention, thecolor conversion table retains pre-converted image data andpost-converted image data. By referencing the color conversion tableusing pre-converted image data, it is possible to attain post-convertedimage data, i.e., correspondence relationship information is stored in aform of table.

[0031] Besides, it is possible to hold correspondence relationshipinformation in such forms as arithmetic expressions, arithmeticparameters, etc. However, in use of a table form, there is an advantagethat operations can be simplified since just referencing is required.

[0032] As stated above, in accordance with the present invention, sincecorrespondence relationship information is held in the color conversiontable, it is relatively easy to carry out image data conversion.

[0033] As to a plurality of correspondence relationships, it is notnecessarily required to provide different kinds of correspondencerelationships.

[0034] Another object of the present invention is to provide an imageprocessing apparatus of attaining results of plural conversionssubstantially.

[0035] According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, a plurality of correspondence relationshipsare realized by changing a degree of correspondence relationshipapplicability.

[0036] In the above-mentioned arrangement of the present invention, aplurality correspondence relationships are set up by changing a degreeof correspondence relationship applicability. For example, according toa degree of applicability, relational association can be made betweenimage data not applied to a certain level of correspondence relationshipand image data applied thereto. In this case, either linear ornon-linear relational association may be made.

[0037] Still more, in use of such a degree of applicability, just asingle correspondence relationship can set up a condition converted byapplying the correspondence relationship and a condition not convertedsubstantially, i.e., it is possible to provide a plurality ofcorrespondence relationships. In this case, more correspondencerelationships can be realized by changing a degree of applicability asrequired.

[0038] As stated above, in accordance with the present invention, it ispossible to rearrange a single correspondence relationship as aplurality of correspondence relationships by changing a degree ofapplicability.

[0039] Another object of the present invention is to provide an imageprocessing method of holding a plurality of correspondence relationshipswhich may be predetermined or specified as required.

[0040] According to the present invention, there is provided an imageprocessing apparatus wherein, the correspondence relationshipinformation holding unit includes the correspondence relationshipspecifying unit in which a plurality of correspondence relationshipsapplicable to the image data are specified in succession as required.

[0041] In the above-mentioned arrangement of present invention, aplurality of correspondence relationships to be applied to the imagedata are specified in succession. There are two kinds of correspondencerelationship elements; a condition to be converted, and an object to beapplied. Either one or both of these kinds of elements may be taken. Inthis case, it is also possible to specify a part of each image andselect a conversion condition for the specified part of image. Stillmore, there maybe provided such an arrangement that a flesh coloradjustment item or a sky-blue color adjustment item is selectable as anoptional function of image processing.

[0042] As stated above, in accordance with the present invention, sinceapplicable correspondence relationships can be specified as required, itis possible to improve a degree of freedom in image processing.

[0043] Another object of the present invention is to provide an imageprocessing apparatus of holding each correspondence relationship foradjusting brightness.

[0044] According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, each correspondence relationship for adjustingbrightness of the image data is held.

[0045] In the above-mentioned arrangement of the present invention, eachcorrespondence relationship for adjusting brightness according to theimage data is held, and brightness of predetermined pixels is adjustedusing the correspondence relationship thus held. For example, on acertain part of image, brightness of pixels thereof is increased.

[0046] As stated above, in accordance with the present invention, it ispossible to adjust brightness on a part of image based on applicablecorrespondence relationship information.

[0047] Another object of the present invention is to provide an imageprocessing apparatus of holding each correspondence relationship foradjusting chromaticity.

[0048] According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, each correspondence relationship for adjustingchromaticity according to the image data is held.

[0049] In the above-mentioned arrangement of the present invention, eachcorrespondence relationship for adjusting chromaticity according to theimage data is held, and chromaticity of predetermined pixels is adjustedusing the correspondence relationship thus held. For example, on a partof image containing flesh color pixels, enhancement is made to provideruddy flesh color.

[0050] As stated above, in accordance with the present invention, it ispossible to adjust chromaticity on a part of image based on applicablecorrespondence relationship information.

[0051] Another object of the present invention is to provide an imageprocessing apparatus of adjusting color vividness.

[0052] According to the present invention, there is provided an imageprocessing apparatus wherein, in the correspondence relationshipinformation holding unit, each correspondence relationship for adjustingcolor vividness according to the image data is held.

[0053] In the above-mentioned arrangement of the present invention, eachcorrespondence relationship for adjusting color vividness according tothe image data is held, and vividness of predetermined pixels isadjusted using the correspondence relationship thus held. For example,to enhance color of any subject on a surrounding background, coloradjustment is made so that the color of the subject becomes more vivid.

[0054] As stated above, in accordance with the present invention, it ispossible to adjust color vividness on a part of image based onapplicable correspondence relationship information.

[0055] If it is distinguished clearly whether a predeterminedcorrespondence relationship is to be applied or not, a conspicuousboundary may take place in an image. Therefore, another object of thepresent invention is to make such a boundary unrecognizable by providingan image processing apparatus for adjusting a level of correspondencerelationship gradually.

[0056] According to the present invention, there is provided an imageprocessing apparatus wherein, in the image data conversion unit, forpixels in a transition region between regions which are different interms of correspondence relationship, a level of correspondencerelationship is adjusted gradually.

[0057] In the above-mentioned arrangement of the present invention, atransition region is provided between regions which are different interms of correspondence relationship, and for pixels in the transitionregion, a level of correspondence relationship is adjusted gradually inimage data conversion, thereby eliminating a stepwise difference in aprocessed image.

[0058] As stated above, in accordance with the present invention, it ispossible to suppress occurrence of a stepwise difference on a boundarybetween regions to which different correspondence relationships areapplied respectively in an image.

[0059] A level of correspondence relationship to be adjusted graduallyin a transition region is selectable as required, and either linear ornon-linear transition may be applied.

[0060] As mentioned in the foregoing description, correspondencerelationship information for image data conversion can be used for coloradjustment. However, as to selection of an element color and a degree ofenhancement, it is required for a human operator to make a decision. Incolor adjustment processing of digital image data, it is difficult toidentify which part will cause conspicuous color deviation in advance.

[0061] For instance, in case of a color image of a building, colordeviation will not be recognized as long as an original color thereof isunknown. However, in case of a flesh color image of a person's face, anoriginal color thereof can be presumed in most cases. Therefore,adjustment of flesh color in an image is performed by the operator sothat natural flesh color is given.

[0062] In color adjustment, there is a problematic factor which isreferred to as a memory color effect in psychology. When a lemon isphotographed as a subject, for example, a color of a photographed lemonimage looks slightly somber even if it meets a color of the actual lemonin measurement using a calorimeter. Agreement between the color of thelemon image and that of the actual lemon cannot be found unless aside-by-side comparison check is made. Due to a memory color effect inpsychology, a person memorizes that a color of a lemon is vivid ingeneral. However, an actual lemon does not have such a vivid color as ismemorized. Therefore, even if a lemon image is corrected to have itsactual color, it is not satisfactory in terms of visual sensation inpsychology. In case of flesh color, green color of tree leaves and bluecolor of sky, a memory color effect tends to occur as stated above.

[0063] When a person conducts adjustment of flesh color, for example, adegree of flesh color enhancement is determined in an appropriate rangethough the flesh color does not exactly match its memory color. If theflesh color is adjusted to exactly match its memory color, deviationwill occur in other colors.

[0064] Therefore, automatic color adjustment cannot be performed just bypredetermining correspondence relationship information. Even inautomatic color adjustment, it is rather difficult to attainsatisfactory result due to a memory color effect, i.e., a person mustreadjust result of automatic color adjustment.

[0065] It is another object of the present invention to provide a coloradjustment device, a color adjustment method and a storage medium forrecording a color adjustment control program for carrying out optimumcolor adjustment processing automatically while taking account of suchfactors as a memory color effect.

[0066] According to the present invention, there is provided a coloradjustment device for performing color separation of color image datafor each predetermined element color and adjusting the color image datain enhancement to provide each desired color in result of color imageoutput delivered for each element color by such a device as an imageoutput device, etc., comprising; the chromaticity judging unit whichdetermines a value of chromaticity of each pixel according to the colorimage data, the object chromaticity pixel statistical calculation unitwhich performs statistical calculation on pixels having chromaticityvalues which have been determined to meet a predefined range ofchromaticity by the chromaticity judging unit, the color adjustmentdegree judging unit which determines a degree of color adjustment so asto eliminate a difference between a predetermined optimum value forpixels meeting the predefined range of chromaticity and a result valueattained in the statistical calculation and for regulating the degree ofcolor adjustment according to an occupancy ratio of pixels subjected tostatistical calculation to the total number of pixels, and the coloradjusting unit which carries out color adjustment of the image dataaccording to the regulated degree of color adjustment.

[0067] In the above-mentioned arrangement of the present invention, avalue of chromaticity of each pixel is determined according to the colorimage data. Since a value of chromaticity represents an absolute ratioin psychophysical color specification which does not depend onbrightness, important parts (objects) in an image can be identifiedaccording to possible ranges of chromaticity. For example, an object canbe distinguished by checking whether its chromaticity value is in apossible chromaticity range of flesh color or green color of treeleaves. Based on this principle of chromaticity, if determinedchromaticity values of pixels are in a predefined chromaticity range,statistical calculation is performed on these pixels. In the aboveexample, if determined chromaticity values of pixels are in the possiblechromaticity range of flesh color, they are subjected to statisticalcalculation. For statistical calculation, an average value, median, etc.may be taken. Statistical calculation on pixels having chromaticity in apredefined range with respect to all the pixels of an image is analogousto a situation that a person judges an average level of flesh color withrespect to flesh-color-like pixels.

[0068] Then, it is necessary to judge whether or not a particular colormatches its memory color. A degree of color adjustment is so determinedas to eliminate a difference between a predetermined optimum value forpixels meeting a predefined range of chromaticity and a result valueattained in the statistical calculation. However, if a degree of coloradjustment thus determined is applied intactly, a flesh color part willbe adjusted to have memory flesh color, for example, but non-allowablecolor deviation will occur on other color parts. Therefore, the degreeof color adjustment thus determined is regulated according to anoccupancy ratio of pixels subjected to the statistical calculation tothe total number of pixels. In this manner, color adjustment is madeproperly so that adjustment of flesh color will not cause an excessivechange in other colors. Thus, the image data is color-adjusted accordingto the regulated degree of color adjustment.

[0069] As stated above, in accordance with the present invention, it ispossible to provide a color adjustment method which enables automaticcolor adjustment processing through chromaticity statistical calculationfor simulating human judgment.

[0070] It is to be understood that a method of performing statisticalcalculation on pixels having chromaticity in a predefined range andreflecting result of statistical calculation in a degree of coloradjustment is practicable in a substantial apparatus for itsimplementation. From this point of view, the present invention providesa color adjustment device for performing color separation of color imagedata for each predetermined element color and adjusting the color imagedata in enhancement to provide each desired color in result of colorimage output delivered for each element color by such a device as animage output device, etc., comprising the steps of; determining a valueof chromaticity of each pixel according to the color image data,performing statistical calculation on pixels having chromaticity valueswhich have been determined to meet a predefined range of chromaticity,determining a degree of color adjustment so as to eliminate a differencebetween a predetermined optimum value for pixels meeting the predefinedrange of chromaticity and a result value attained in the statisticalcalculation, regulating the degree of color adjustment according to anoccupancy ratio of pixels subjected to the statistical calculation tothe total number of pixels, and carrying out color adjustment of theimage data according to the regulated degree of color adjustment.

[0071] As an example of practicing the present invention, it may beembodied in color adjustment control software for a color adjustmentdevice. In this case, according to the present invention, there isprovided a software storage medium for recording the color adjustmentcontrol software. From this viewpoint, the present invention provides astorage medium for recording a color adjustment control program forperforming color separation of color image data for each predeterminedelement color on a computer and adjusting the color image data inenhancement to provide each desired color in result of color imageoutput delivered for each element color by such a device as an imageoutput device, etc., the color adjustment control program comprising thesteps of; determining a value of chromaticity of each pixel according tothe color image data, performing statistical calculation on pixelshaving chromaticity values which have been determined to meet apredefined range of chromaticity, determining a degree of coloradjustment so as to eliminate a difference between a predeterminedoptimum value for pixels meeting the predefined range of chromaticityand a result value attained in the statistical calculation, regulatingthe degree of color adjustment according to an occupancy ratio of pixelssubjected to the statistical calculation to the total number of pixels,and carrying out color adjustment of the image data according to theregulated degree of color adjustment. In this case, the samefunctionalities as in the foregoing description can be provided.

[0072] In determining chromaticity of each pixel of color image data,chromaticity conversion is performed taking account of a color spacecoordinate scheme used for original color image data. In this case,chromaticity is used intactly if it is a direct element, or chromaticityis converted if it is not a direct element. For conversion, a conversiontable or a conversion expression may be used. In this case, it is notnecessarily required to attain an accurate value in result, i.e., anerror may be contained if its adverse effect is insignificant.

[0073] When determined chromaticity values of pixels are in a predefinedchromaticity range, statistical calculation is performed on thesepixels. In this case, it is not necessarily required to make analternative-choice judgment, i.e., it is not necessarily required todetermine whether the predefined chromaticity range is satisfied or not.Instead, statistical calculation may be made while changing a weightfactor.

[0074] A variety of statistical calculation approaches may be used inapplication.

[0075] Another object of the present invention is to provide a coloradjustment method which is advantageous in the amount of calculation.

[0076] According to the present invention, there is provided a coloradjustment device wherein: in object chromaticity pixel statisticalcalculation unit, an average value of pixels judged to be object pixelsis calculated for each element color of color image data; and in coloradjustment degree judging unit, an optimum value of each element coloris provided for color image data meeting the predefined range ofchromaticity, and a degree of color adjustment is determined accordingto the optimum value of each element color.

[0077] In the above-mentioned arrangement of the present invention,statistical calculation is performed for each element color of colorimage data to determine an average value of pixels judged to be objectpixels, i.e., statistical calculation is carried out using each elementcolor of color image data instead of chromaticity statisticalcalculation. On the other hand, an optimum value of each element coloris provided for color image data meeting the predefined range ofchromaticity. In judgment of a degree of color adjustment, comparisonfor each element color is performed, and a difference attained is usedas a degree of element color adjustment.

[0078] In accordance with the present invention, it is judged fromchromaticity whether or not an objet is applicable to statisticalcalculation. Since statistical calculation is performed for each elementcolor of color image data, statistical result can be used easily insubsequent arithmetic operations.

[0079] Besides, it is possible to use median calculation, standarddeviation calculation, etc. though the amount of calculation increasesin statistics.

[0080] A range of chromaticity to be subjected to statisticalcalculation can be changed as required.

[0081] Another object of the present invention is to provide a coloradjustment device which is suitable for memory color adjustment.

[0082] According to the present invention, there is provided a coloradjustment device wherein, in the object chromaticity pixel statisticalcalculation unit, statistical calculation is performed on objectchromaticity pixels in a possible range of chromaticity in terms ofmemory color in psychology.

[0083] In the above-mentioned arrangement of the present invention,statistical calculation is performed on object chromaticity pixels whichmeet a possible range of chromaticity in terms of memory color inpsychology. In this case, it is not necessarily required to attainstatistical result using just one possible range of chromaticity. It ispossible to carry out a plurality of statistical calculationsrespectively by specifying plural possible chromaticity ranges.

[0084] In accordance with the present invention, since calculation isperformed on pixels corresponding to memory color, a degree of coloradjustment is regulated according to the number of pixels, therebymaking it possible to form a judgment similar to that by a person.

[0085] In judgment of a degree of color adjustment, a degree of coloradjustment is so determined as to eliminate a difference between apredetermined optimum value for pixels meeting a predefined range ofchromaticity and a result value attained in the statistical calculation,and the degree of color adjustment is regulated according to anoccupancy ratio of pixels subjected to statistical calculation to thetotal number of pixels. In this case, it is not necessarily required tocarry out these two steps of processing. It is just required to providethe same result of processing. For example, if a difference between apredetermined optimum value for pixels meeting a predefined range ofchromaticity and a result value attained in statistical calculation isobtained in a step of statistical calculation, it may be reflected inresult of statistical calculation. If an occupancy ratio of pixelssubjected to statistical calculation to the total number of pixels isobtained in the step of statistical calculation, it may be reflected inresult of statistical calculation. Sill more, it is not necessarilyrequired to perform subtraction to attain a difference between apredetermined optimum value and a result value in statisticalcalculation. It is just required to include any proper calculationinherently.

[0086] A degree of color adjustment may also be determined in a varietyof forms.

[0087] Another object of the present invention is to provide a coloradjustment device of determining a degree of color adjustment.

[0088] According to the present invention, there is provided a coloradjustment device wherein, in the color adjustment degree judging unit,atone curve representing input-output relationship information is usedfor changing a degree of element color enhancement, and a tone curve isformed according to the degree of color adjustment.

[0089] In the above-mentioned arrangement of the present invention, atone curve is formed according a degree of color adjustment, and adegree of element color enhancement is changed according to the tonecurve thus formed so that input-output relationship information on eachelement color is represented by the tone curve.

[0090] In accordance with the present invention, since a tone curve isused, it is easy to add adjusted conditions of other element colors froman overall point of view. In this case, a memory color part is adjustedtoward an ideal color level and adjacent color regions are adjustedgradually in a continuous manner, thereby preventing occurrence ofpseudo-contouring due to tone jumping.

[0091] The above and other objects, features and advantages of thepresent invention will become more apparent from the followingdescription of embodiments with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0092]FIG. 1 is a block diagram showing an image processing apparatus ina preferred embodiment of the present invention;

[0093]FIG. 2 is a block diagram showing concrete examples of hardwarefor the image processing apparatus according to the present invention;

[0094]FIG. 3 is a schematic block diagram showing another example ofapplication of the image processing apparatus according to the presentinvention;

[0095]FIG. 4 is a schematic block diagram showing another example ofapplication of the image processing apparatus according to the presentinvention;

[0096]FIG. 5 is a flowchart of image processing in the image processingapparatus according to the present invention;

[0097]FIG. 6 is a diagram showing a state in which a processing objectpixel is moved;

[0098]FIG. 7 is a diagram showing a display window for specifying aprocessing object area and a processing item;

[0099]FIG. 8 is a diagram showing that two processing object areas areselected;

[0100]FIG. 9 shows a region reference table used for storing specifiedregions and image processing objects;

[0101]FIG. 10 is a diagram showing a window for selecting a processingobject and a processing item;

[0102]FIG. 11 is a diagram showing rectangular regions specified withchromaticity “x-y”;

[0103]FIG. 12 is a diagram showing that center points are specified withchromaticity “x-y”;

[0104]FIG. 13 is a flowchart of applicability setting;

[0105]FIG. 14 is a graph showing a change in applicability when a centerpoint is specified with chromaticity “x-y”;

[0106]FIG. 15 is a graph showing a luminance distribution range inexpansion;

[0107]FIG. 16 shows a conversion table for expanding a luminancedistribution range;

[0108]FIG. 17 is a graph showing a conversion relationship for expandinga luminance distribution range;

[0109]FIG. 18 is a graph showing a conceptual condition for increasingluminance by γ-curve correction;

[0110]FIG. 19 is a graph showing a conceptual condition for decreasingluminance by γ-curve correction;

[0111]FIG. 20 is a graph showing a correspondence relationship ofluminance subjected to γ-correction;

[0112]FIG. 21 is a diagram showing an unsharp mask comprising 5×5pixels;

[0113]FIG. 22 is a graph showing a statistical condition of saturationdistribution;

[0114]FIG. 23 is a graph showing a relationship between minimumsaturation ‘A’ and saturation index ‘S’;

[0115]FIG. 24 is a diagram showing a display window which contains anarea to be processed in entire image processing and an area to beprocessed in partial image processing;

[0116]FIG. 25 is a block diagram showing a color adjustment device in apreferred embodiment of the present invention;

[0117]FIG. 26 is a flowchart of color adjustment processing in the coloradjustment device according to the present invention;

[0118] FIGS. 27(a)-(c) are diagrammatic illustrations showing tone curesfor converting gradation data with predetermined degrees of enhancement;

[0119]FIG. 28 is a flowchart of color adjustment processing in amodified embodiment according to the present invention;

[0120]FIG. 29 is a diagram showing a window for selecting a coloradjustment object for the color adjustment processing; and

[0121]FIG. 30 is a diagram illustrating an original photographic imageto be subjected to the color adjustment processing.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0122] The present invention will now be described in detail by way ofexample with reference to the accompanying drawings.

[0123]FIG. 1 shows a block diagram of an image processing scheme usingan image processing apparatus in a preferred embodiment of the presentinvention, and FIG. 2 presents a schematic block diagram illustrating aconcrete example of a hardware configuration.

[0124] Referring to FIG. 1, an image input device 10 suppliesphotographic image data containing dot-matrix pixels of a photographicor picture image to an image processing apparatus 20, on which an objectapplicable to image processing and conditions thereof are specified andthen image processing is carried out for object pixels. Thereafter, theimage processing apparatus 20 delivers the image data thus processed toan image output device 30, on which the processed image is output in adot-matrix pixel form.

[0125] As shown in FIG. 2, a concrete example of the image input device10 is a scanner 11, a digital still camera 12 or a video camcorder 14, aconcrete example of the image,processing apparatus 20 is a computersystem comprising a computer 21, a hard disk unit 22, a keyboard 23, amouse 27, a CD-ROM drive unit 24, a floppy disk drive unit, 25, a modem26, etc., and a concrete example of the image output device 30 is aprinter 31 or a display monitor 32. In the present preferred embodiment,photographic image data is most suitable as input data since objetpixels are specified for correcting deficiencies in an image and imageprocessing is performed according to a predetermined correspondencerelationship. The modem 26 can be connected to a public communicationline for downloading software and data from an external network throughthe public communication line.

[0126] In the present preferred embodiment, the scanner 11 or digitalstill camera 12 serving as the image input device 10 provides RGB (red,green, blue) gradation image data, the printer 31 serving as the imageoutput device 30 requires input of CMY (cyan, magenta, yellow) or CMYK(cyan, magenta, yellow, black) binary gradation data, and the displaymonitor 32 requires input of RGB gradation data. In the computer 21, aprinter driver 21 b for the printer 31 and a display driver 21 c for thedisplay monitor 32 are run under control of an operating system 21 a.

[0127] Furthermore, an image processing application 21 d is controlledby the operating system 21 a for execution of processing thereof, andthe image processing application 21 d carries out predetermined imageprocessing in conjunction with the printer driver 21 b or the displaydriver 21 c as required. Therefore, the computer 21 serving as the imageprocessing apparatus 20 is used for receiving RGB gradation data andgenerating optimally image-processed RGB gradation data through imageevaluation for output onto the display monitor 32 by means of thedisplay driver 21 c. The computer 21 is also used for converting RGBgradation data into CMY (or CMYK) binary data for output onto theprinter 31 by means of the printer driver 21 b.

[0128] As mentioned above, the computer system is arranged between theimage input device and the image output device for carrying out imageevaluation and image processing in the present preferred embodiment. Thecomputer system is not always-required, however It is to be understoodthat the invention is applicable to a variety of image data processingapparatuses. As shown in FIG. 3, for example, there may be provided sucha system arrangement that an image processing apparatus for carrying outimage processing for each predetermined applicable object isincorporated in a digital still camera 12 a and converted image data isdisplayed on a display monitor 32 a or printed on a printer 31 a. Stillmore, as shown in FIG. 4, for a printer 31 b capable of receiving imagedata and printing images without using a computer system, there may beprovided such an arrangement that image data is input from a scanner 11b, a digital still camera 12 b or modem 26 b to the printer 31 b, inwhich image processing is carried out through judgment on whether or noteach pixel belongs to each applicable object.

[0129] In the embodiments mentioned above, image evaluation and imageprocessing are implemented by means of an image processing programflowcharted in FIG. 5, which is run on the computer 21, for example.Referring to FIG. 5, the image processing program is executed throughthe following steps: At the first step S100, an applicable object isspecified for each image processing. At steps S110 to S140, apredetermined image processing operation is performed by moving anobject pixel for each dot-matrix pixel as shown in FIG. 6. Morespecifically, at step S110, it is judged whether each object pixel isapplicable to image processing conditions specified at S100. At stepS120, image data is converted according to the result of judgment. Inthis conversion, image processing is accomplished in effect. At stepsS130 and S140, each object pixel is moved as required. The followingdescribes this image processing flow in further detail.

[0130] Depending on image processing conditions, it is determined whichpixels are to be subjected to image processing. In the present preferredembodiment, a part of image to be processed is specified as shown inFIGS. 7 and 8. Referring to FIG. 7, there are provided an operation area41 along a top side of a window 40 for window frame operation, a displayarea 42 at a center part of the window 40, and a processing menu area 43at a bottom side of the window 40 for image processing menu items.

[0131] An image to be processed is displayed on the display area 42, arectangular region is specified using the mouse 27, and an imageprocessing menu item is selected on the processing menu area 43 usingthe mouse 27. The processing menu area 43 contains executable imageprocessing items including ‘CONTRAST’ adjustment, ‘BRIGHTNESS’adjustment, ‘SHARPNESS’ adjustment and ‘SATURATION’ adjustment items,each of which is provided with arrows for specifying a degree ofadjustment. For instance, clicking an up-arrow of ‘CONTRAST’ adjustmentitem with the mouse 27 specifies image processing for increasingcontrast, and clicking a down-arrow thereof specifies image processingfor decreasing contrast.

[0132] For each specified region of image, a plurality of imageprocessing items may be selected. When ‘LIST DISPLAY/EDIT’ item on theprocessing menu area 43 is clicked, information on a selected imageprocessing item is displayed as shown in a region reference tableexemplified in FIG. 9, i.e., an upper left corner coordinate point and alower right corner coordinate point of each specified region, aspecified kind of image processing, and a specified level of adjustmentare displayed. A level value indicates a degree of image processingadjustment, which can be increased/decreased by clicking anup/down-arrow of each image processing item on the processing menu area43 as many times as required. A kind-of-processing code indicates aparticular kind of image processing as follows: In the example shown inFIG. 9, the CONTRAST adjustment item is indicated as ‘1’, the BRIGHTNESSadjustment item is indicated as ‘2’, the SHARPNESS adjustment item isindicated as ‘3’, and the SATURATION adjustment item is indicated as‘4’. The kinds of image processing and levels of adjustment will bedescribed in more detail together with description of image processingconditions. For removing a image processing item from the regionreference table, it is just required to highlight the image processingitem by clicking with the mouse and press a DELETE key on the keyboard23. Besides, the contents of the region reference table can be edited inother manners similar to those in ordinary application software.

[0133] In the present preferred embodiment, since the image processingapplication 21 d is executed at an application level of the operatingsystem 21 a, an image be processed can be displayed in the window 40 anda region of interest can be specified in it. However, in an instancewhere the same image processing functionality is implemented at a driverlevel of the operating system 21 a, e.g., in the printer driver 21 b, animage to be processed may not be displayed in a window.

[0134]FIG. 10 shows an example in which an object of image processing isspecified regardless of whether windowing display is provided or not.When the printer driver 21 b is made active, an window 50 is selectableas an option. While a part of image to be processed is specified in theexamples shown in FIG. 7 to 9, pixels to be processed are selected byspecifying image chromaticity this example. The following describeschromaticity used in the invention.

[0135] As an example of representation in chromaticity, “x-y”chromaticity calculation is performed below. Assuming that RGB gradationdata containing pixels to be processed in RGB color scheme is (R, G, B),the following expressions are given.

r=R/(R+G+B)  (1)

g=G/(R+G+B)  (2)

[0136] Then, in XYZ color scheme, the following correspondencerelationship is set up between chromaticity coordinates “x” and “y”.

x=(1.1302+1.6387r+0.6215g)/(6.7846−3.0157r−0.3857g)  (3)

y=(0.0601+0.9399r+4.5306g)/(6.7846−3.0157r−0.3857g)  (4)

[0137] Since a value of chromaticity represents an absolute ratio inpsychophysical color specification which does not depend on brightness,it is possible to identify an object to which the pixels of interestbelong according to chromaticity thereof.

[0138] For example, in an object image of a person, it is necessary toextract pixels in a flesh color part. A value of chromaticity of fleshcolor is in a range expressed below.

0.35<×<0.40  (5)

0.33<y<0.36  (6)

[0139] Hence, if a determined chromaticity value of each pixel is withinthe range indicated above, it can be assumed that the pixel of interestcorresponds to a point of skin of a person's image in most cases.Similarly, as to such object images as blue sky and green leaves oftrees, chromaticity is also applicable in judgment. Regardless ofbrightness, pixels of blue sky and green leaves of trees can beidentified approximately. FIG. 11 shows a distribution relationship of“x-y” chromaticity, in which there are indicated applicable rectangularregions corresponding to flesh color, sky-blue color and green color ofleaves. As shown in FIG. 10, there may be provided such:an arrangementthat a range value of “x-y” chromaticity distribution can be specifieddirectly for individual definition.

[0140]FIG. 12 shows a technique for specifying a center point indetermination of each applicable region. In case of flesh color, acenter point of its, applicable region is found as expressed below usingEquations (5) and (6).

x=0.375  (7)

y=0.345  (8)

[0141] Then, with reference to this center point, it is checked whethera value of chromaticity is within a range of a predetermined radius ornot. If this condition is satisfied, it is judged that each pixel ofinterest belongs to a particular applicable region.

[0142] In the window 50 selectable using an optional function of theprinter driver 21 b, there are provided adjustment items which indicatepractical examples corresponding to predefined chromaticity ranges to bespecified for an object so that a human operator can easily adjust acolor of blue sky or green leaves of trees as desired. In addition,there are provided arrow buttons which are used for increasing ordecreasing chromaticity of pixels of interest.

[0143] In the example shown in FIG. 10, a range of pixels is specifiedwith chromaticity and it is determined whether the chromaticity is to beenhanced or not in image processing. However, it is not always necessaryto let chromaticity-based pixel selection be consistent with imageprocessing to be performed. For instance, in image processing, it isallowed to increase brightness of flesh color pixels or contrast ofgreen color pixels. Namely, chromaticity is used only as an indexindicating whether pixels of interest are to be processed or not. Inthis example, chromaticity is used just for the purpose of pixelselection in image processing.

[0144] As described above, step S100 at which an applicable object isspecified for each image processing comprises the unit for specifying acorrespondence relationship. However, it is to be understood that stepS100 is not limited to this function and may be modified to provide theunit for specifying a part of image or a value of chromaticity asrequired. There may also be provided such an arrangement that otherfactors including brightness and sharpness are specified. As shown inFIG. 9, the region reference table for indicating a specified region andholding its image processing in a correspondence relationship is storedin a memory area of the computer system in the present preferredembodiment. It will be obvious to those skilled in the art that theregion reference table is used to provide the unit for holding acorrespondence relationship in combination with a color conversion tablefor image processing (to be described later).

[0145] At step S110 and the subsequent steps, each object pixel is movedand it is judged whether each object pixel is applicable to a specifiedoperation of image processing as mentioned above. For checking whethereach object pixel is to be subjected to image processing or not, theregion reference table shown in FIG. 9 is used. A coordinate location ofeach object pixel is compared with upper left corner and lower rightcorner coordinates indicated in each field of the region referencetable. If the coordinate location of each object pixel meets acoordinate value indicated in any field, it is judged that the objectpixel of interest is applicable to image processing. If it does not meetany coordinate value indicated in any field, it is judged that theobject pixel of interest is not applicable to image processing. In casethat chromaticity is used for judgment as shown in FIG. 10, object pixelchromaticity is calculated using Equations (1) to (4), and it is judgedwhether a calculated value is within a chromaticity range selected usingan optional function of the printer driver 21 b.

[0146] Whether an object is applicable to image processing or not isjudged according to a part of image or chromaticity as mentioned above.However, if an alternative-choice judgment is made to determine whethera specific range is satisfied or not, any visual stepwise difference isprone to occur in an adjacent region. For instance, if brightness of acertain rectangular region is increased while brightness of an immediateoutside part thereof is not increased, a stepwise difference inbrightness occurs on a boundary therebetween, resulting in therectangular region being distinguishable clearly.

[0147] To circumvent this adverse effect, the present preferredembodiment uses a parameter of image processing applicability k.Referring to FIG. 13, if an object is judged to belong to a specifiedregion at step Sill, applicability k is set to ‘1’ at step S112. For atransition region on a circumference of the specified region,applicability k is changed in a range of ‘0’ to ‘1’. More specifically,if the object is judged to be within the transition region at step S113,applicability k is adjusted toward ‘1’ at step S114 in case the objectis near the specified region or it is adjusted toward ‘0’ in case theobject is apart from the specified region. If the object is judged notto belong to the specified region nor the transition region,applicability k is set to ‘0’ at step S115. While a center point ofchromaticity is specified as exemplified in FIG. 12, there may beprovided such an arrangement as shown in FIG. 14. Referring to FIG. 14,applicability k is set to ‘1’ for an image processing objectcorresponding to radius ‘r0’, and under condition that a transitionregion is provided in a range of radius ‘r0’ to ‘r1’, applicability k isgradually adjusted toward ‘0’ in a circumferential region.

[0148] At step S110, an applicable correspondence relationship isjudged, and at steps S111 to S115, applicability k is determined asmentioned above. According to the present preferred embodiment, there isprovided the unit for judging a correspondence relationship usingsoftware processing including such procedural steps as steps S110 toS115 mentioned above and hardware for implementation thereof. Whileapplicability k is determined as a result of judgment in each imageprocessing in the present preferred embodiment, it will be obvious tothose skilled in the art that an alternative-choice judgment is alsofeasible in application.

[0149] At step S120, image processing is carried out for each objectpixel according to applicability k. The following describes morespecific conditions of image processing in practicing the presentinvention.

[0150] Contrast indicates a width of luminance in an entire image, andinmost cases of contrast adjustment, it is desired to increase a widthof contrast. Referring to FIG. 15, there is shown a histogram ofstatistical luminance distribution of pixels of a certain image. In caseof a solid-line curve of narrow distribution, a difference in luminanceamong bright and dark pixels is relatively small. In case of adot-dashed-line curve, of broad distribution, a difference in luminanceamong bright and dark pixels is relatively large, which signifies that awidth of contrast is increased. Referring to FIG. 17, there is shown agraph indicating a luminance conversion operation for enhancingcontrast. Assuming that the following equation holds for a relationshipbetween original unconverted luminance y and converted luminance Y,

Y=ay+b  (9)

[0151] Through conversion under condition “a>1”, a difference between amaximum value of luminance ‘ymax’ and a minimum value of luminance‘ymin’ is increased, resulting in a broad distribution of luminance asshown in FIG. 15. In this case, it is preferred to determine slope ‘a’and offset ‘b’ according to luminance distribution. For example, thefollowing equations are given:

a=255/(ymax−ymin)  (10)

b=−a·ymin or 255−a·ymax  (11)

[0152] Under the conditions indicated above, a certain narrow luminancedistribution can be expanded to a reproducible extent. However, if it isexpanded to an extreme end of a reproducible range,, highlights may beblown out to white or shadows may be plugged up to black. To preventthis, a non-expand able margin corresponding to a luminance value ofapprox. ‘5’ is provided at each of the upper and lower limits of thereproducible range. Resultantly, conversion parameters are expressed asfollows:

a=245/(ymax−ymin)  (12)

b=5−a·ymin or 250−a·ymax  (13)

[0153] In this case, conversion is not performed in ranges where “y<5”and “y>250”.

[0154] In image data conversion, it is not required to performcalculation each time. If a luminance value range is ‘0’ to ‘255’, aresult of conversion is predetermined for each luminance value and aconversion table is prepared as shown in FIG. 16. In this case, theconversion table is usable just for luminance. In an instance whereimage data contains direct elements of luminance, the conversion tablecan be used. Contrarily, in an instance where image data contains onlyindirect elements of luminance, the conversion table cannot be used. Inmost computer systems, red, green and blue elements of image data areindicated in gradation of brightness, i.e., color gradation data (R, G,B) is used. Since such gradation data (R, G, B) does not provide directvalues of luminance, it is necessary to perform conversion to Luv colorspace scheme for determining luminance. This method is, however,disadvantageous since a large amount of calculation is required.Therefore, the following conversion expression is used, which iscommonly adopted in television signal processing for directlydetermining luminance from RGB data:

y=0.30R+0.59G+0.11B  (14)

[0155] On a principle that linear conversion can be made betweengradation data and luminance y as indicated above, Equation (9) isapplicable to a relationship between original unconverted gradation data(R0, G0, B0) and converted gradation data (R1, G1, B1). Then, thefollowing expressions are given:

R 1=aR 0+b  (15)

G 1=aG 0+b  (16)

B 1=aB 0+b  (17)

[0156] Consequently, the conversion table shown in FIG. 16 is applicableto gradation data conversion.

[0157] The following describes an image processing technique foradjusting brightness. As in the above-mentioned case of contrastadjustment, a histogram of brightness distribution is assumed. Referringto FIG. 18, a solid-line curve indicates a luminance distribution whichhas its peak inclined toward a dark level. In this case, the peak ofentire distribution is shifted toward a bright side as indicated by abroken-line curve. Referring to FIG. 19, a solid-line curve indicates aluminance distribution which has its peak inclined toward a bright side.In this case, the peak of entire distribution is shifted toward a darkside as indicated by a broken-line curve. In these cases, linearluminance conversion as shown in FIG. 17 is not performed, but γ-curveluminance conversion is performed as shown in FIG. 20.

[0158] In γ-curve correction, entire brightness is increased when “γ<1”,and it is decreased when “γ>1”. A degree of this correction can beadjusted gradually by clicking an up-arrow or down-arrow of theBRIGHTNESS adjustment item on the processing menu area 43 shown in FIG.7 as many time as required at step S100.

[0159] As in contrast adjustment, it is also possible to set up a valueof γ automatically. As a result of our various experiments, it has beenfound that the following approach is advantageous: In a luminancedistribution, median ‘ymed’ is predetermined. If it is less than ‘85’,an image of interest is judged to be too dark and γ correction is madeaccording to a γ value indicated below.

γ=ymed/85  (18)

[0160] or,

γ=(ymed/85) ** (1/2)  (19)

[0161] Note, however, that even if “γ<0.7”, a value of γ is set to 0.7forcedly. Unless this kind of limit is provided, a night-scene imageappears to be of daytime. If an image is brightened excessively, itbecomes whitish entirely, resulting in low contrast. In this case, it ispreferred to perform such a processing operation as enhancement ofsaturation in combination.

[0162] If median ‘ymed’ is larger than ‘128’, an image of interest isjudged to be too bright and γ correction is made according to a γ valueindicated below.

γ=ymed/128  (20)

[0163] or,

γ=(ymed/128) ** (1/2)  (21)

[0164] In this case, a limit is also provided so that a value of γ isset to 1.3 forcedly even if “γ>1.3” for the purpose of preventing theimage of interest from becoming too dark.

[0165] For this γ correction, it is preferred to provide such aconversion table as shown in FIG. 16.

[0166] In edge enhancement processing for adjusting sharpness of animage, with respect to original non-enhanced luminance Y of each pixel,enhanced luminance Y′ is calculated as expressed below.

Y′=Y+Eenhance·(Y−Yunsharp)  (22)

[0167] where ‘Eenhance’ indicates a degree of edge enhancement, and‘Yunsharp’ indicates unsharp-mask processing for each pixel of imagedata. The following describes the unsharp-mask processing: Referring toFIG. 21, there is shown an example of an unsharp mask 60 comprising 5×5pixels. The unsharp mask 60 is used in summation in such a manner that acenter value of ‘100’ is assigned as a weight to a processing objectpixel ‘Y(x,y)’ in dot-matrix image data and a weight corresponding to avalue in each array box of the unsharp mask 60 is assigned to eachcircumferential pixel thereof. In use of the unsharp mask 60, summationis performed based on the following expression: $\begin{matrix}{{{Yunsharp}\quad \left( {x,y} \right)} = {\left( {1/396} \right){\sum\limits_{ij}\left( {{Mij} \times {Y\left( {{x + i},{Y + j}} \right)}} \right)}}} & (23)\end{matrix}$

[0168] In Equation (23), ‘396’ indicates a total value of weightingfactors. For an unsharp mask having a different size, a total of valuesin array boxes thereof is indicated. ‘Mij’ represents a weighting factorindicated in each array box of an unsharp mask, and ‘Y(x,y) ’ representseach pixel of image data. In the unsharp mask 60, ‘i’ and ‘j’ indicatecoordinates on horizontal and vertical axes thereof.

[0169] Edge enhancement calculation based on Equation (22) provides thefollowing functional meaning: ‘Yunsharp(x,y)’ is the result of additionin which a weight assigned to each circumferential pixel is reduced withrespect to a pixel of interest, and accordingly an image is unsharpenedthrough processing. Such an unsharpening operation is functionallyequivalent to low-pass filtering. Therefore, “Y(x,y) −Yunsharp(x,y)”signifies that a low-frequency component is subtracted from each oforiginal components, i.e., it is functionally equivalent to high-passfiltering. If a high-frequency component subjected to high-passfiltering is multiplied by a degree of edge enhancement ‘Eenhance’ and aresult value of this multiplication is added to “Y(x,y)”, it means thata high-frequency component is increased in proportion to the degree ofedge enhancement ‘Eenhance’. Thus, edge enhancement is accomplished.Since edge enhancement is required only for an edge part of an image, itis possible to reduce the amount of processing substantially byperforming only in case that there is a large difference betweenadjacent pixels of image data.

[0170] In this case, the degree of edge enhancement ‘Eenhance’ can beadjusted by clicking an up-arrow or down-arrow of the SHARPNESSadjustment item on the processing menu area 43 shown in FIG. 7 as manytimes as required at step S100. Still more, it is possible to set up thedegree of edge enhancement ‘Eenhance’ automatically.

[0171] At an edge part of an image, a difference in gradation dataincreases between adjacent pixels. This difference represents a gradientof luminance, which is referred to as a degree of edging. In an image, adegree of variation in luminance can be calculated by determining eachof horizontal-direction and vertical-direction vector components.Although an object pixel in an image containing dot-matrix pixels isadjacent to eight pixels, a degree of variation is determined only withrespect to adjacent pixels in horizontal and vertical directions for thepurpose of simplifying calculation. Summation is performed on lengthvalues of respective vectors to represent a degree of edging ‘g’ for theobject pixel of interest, and a sum value of edging degree is divided bythe number of pixels to attain an average value. Assuming that thenumber of pixels is indicated as ‘E(I)pix’, a degree of sharpness ‘SL’of an object image can be calculated as expressed below.

SL=Σ|g|/E(I)pix  (24)

[0172] In this case, as a value of ‘SL’ decreases, the degree ofsharpness becomes lower (blurring). As a value of ‘SL’ increases, thedegree of sharpness becomes higher (clearer imaging).

[0173] Since sharpness of an image depends on a visual sensation of anindividual person, a degree of sharpness ‘SL’ is determined similarlyusing image data which has optimum sharpness attained experimentally. Avalue thus determined is set as an ideal level of sharpness ‘SLopt’, anda degree of edge enhancement ‘Eenhance’ is determined as expressedbelow.

Eenhance=ks·(SLopt-SL) ** (1/2)  (25)

[0174] where coefficient ‘ks’ varies with a size of image. In case thatimage data contains ‘height’ dots and ‘width’ dots in vertical andhorizontal directions respectively, coefficient ‘ks’ can be determinedas indicated below.

ks=min (height, width) /A  (26)

[0175] where ‘min (height, width)’ indicates the number of ‘height’ dotsor the number of ‘width’ dots, whichever is smaller, and ‘A’ is aconstant value of ‘768’.

[0176] It is to be understood that the above value has been attainedfrom experimental results and may be altered as required. Basically, asan image size increases, it is advisable to increase the degree of edgeenhancement.

[0177] In the above-mentioned fashion, edge enhancement processing canbe carried out in manual or automatic setting.

[0178] The following describes an image processing technique foradjusting saturation. In case of saturation adjustment using asaturation enhancement parameter ‘Sratio’, the parameter can be changedas required for such image data that has saturation parameters. Forattaining a saturation value from gradation data containing RGBcomponent values only, it is primarily necessary to perform conversionto a color space scheme in which saturation values are used as directcomponent values. However, in this processing, RGB image data isconverted into Luv-space image data, and then after saturationenhancement, it is re-converted into RGB image data again, resulting inan increase in the amount of calculation. Therefore, RGB gradation datais directly subjected to saturation enhancement.

[0179] In RGB color space scheme in which components are represented byhue component values having approximately equivalent relationships,condition “R=G=B” signifies gray without saturation. It is thereforeconsidered that a minimum value of each component indicates just adecrease in saturation without affecting hue of each pixel. Based onthis principle, saturation can be enhanced by subtracting a minimumvalue of each component from all the component values and increasing aresultant difference value of subtraction.

[0180] If a component value of blue (B) is minimum in RGB gradation datacontaining red, green and blue components (R, G, B), conversion isperformed as shown below using the saturation enhancement parameter‘Sratio’.

i R′=B+(R−B)×Sratio  (27)

G′=B+(G−B)×Sratio  (28)

B′=B  (29)

[0181] Thus, since there is no need to perform conversion andre-conversion between RGB color space scheme and Lub space scheme, atime required for processing can be reduced substantially. While thepresent preferred embodiment uses a technique of subtracting a minimumcomponent value from another component value as to a non-saturatedcomponent, a different conversion equation may be used for subtracting avalue of a non-saturated component. In case that just a minimum value issubtracted as in Equations (27) to (29), the amount of processing isrelatively small since multiplication or division is not involved.

[0182] In saturation adjustment using Equations (27) to (29),satisfactory conversion can be performed but there is a tendency that animage becomes brighter entirely due to an increase in luminance whensaturation is enhanced. Therefore, conversion is performed for adifference value attained by subtracting a value corresponding toluminance from each component value. Assuming that saturationenhancement is expressed as shown below,

R′=R+ΔR  (30)

G′=G+ΔG  (31)

B′=B+ΔB  (32)

[0183] Each of the above operands ΔR, ΔG and ΔB can be determinedaccording to a difference value with respect to luminance as shownbelow.

ΔR=(R−Y)×Sratio  (33)

ΔG=(G−Y)×Sratio  (34)

ΔB=(B−Y)×Sratio  (35)

[0184] Hence, conversion can be carried out as expressed below.

R′=R+(R−Y)×Sratio  (36)

G′=G+(G−Y)×Sratio  (37)

B′=B+(B−Y)×Sratio  (38)

[0185] As to retention of luminance, the following equations areapplicable:

Y′=Y+Δy  (39)

[0186] $\begin{matrix}\begin{matrix}{{\Delta Y} = \quad {{0.30\Delta \quad R} + {0.59\Delta \quad G} + {1.11\Delta \quad B}}} \\{= \quad {{Sratio}\quad \left\{ {\left( {{0.3R} + {0.59G} + {0.11B}} \right) - Y} \right\}}} \\{= \quad 0}\end{matrix} & (40)\end{matrix}$

[0187] In case of input of gray (R=G=B), condition “luminance Y=R=G=B”is set up, resulting in condition “operand ΔR=ΔG=ΔB=0”. Thus, no coloris given in case of non-saturation. In use of Equations (36) to (38),luminance can be retained and an image does not become brighter entirelyeven when saturation is enhanced.

[0188] In this case, the saturation enhancement parameter ‘Sratio’ canbe adjusted by clicking an up-arrow or down-arrow of the SATURATIONadjustment item on the processing menu area 43 shown in FIG. 7 as manytimes as required at step 100. Still more, it is possible to set up thesaturation enhancement parameter “Sratio” automatically.

[0189] A value of pixel saturation can be determined in a simplifiedmanner. For this purpose, a substitute value ‘X’ of saturation iscalculated as shown below.

X=|G+B−2×R|  (41)

[0190] Essentially, a value of saturation becomes ‘0’ under condition“R=G=B”, and it becomes maximum when a single color of red, green andblue is given or two colors thereof are mixed at a predetermined ratio.Based on this nature, it is possible to represent a value of saturationdirectly. Yet, using the simple Equation (41), a maximum value ofsaturation is provided for a single color of red or a mixture color ofcyan produced by mixing green and blue, and a value of saturation isindicated as ‘0’ when each component is uniform. For a single color ofgreen or blue, approximately half a level of maximum is provided. It isobvious that substitution into the following equations is also possible.

X′=|R+B−2×G|  (42)

X″=|G+R−2×B|  (43)

[0191] In a histogram distribution as to the substitute value ‘X’ ofsaturation, saturation levels are distributed in a range of minimum ‘0’to maximum ‘511’ as schemed in FIG. 22. Then, according to thestatistical saturation distribution shown in FIG. 22, a saturation index‘S’ of an image of interest is determined. In the saturationdistribution, a range of upper ‘16%’ is defined, and a minimumsaturation value ‘A’ in the defined range is taken to representsaturation of the image of interest.

[0192] If A<92, then

S=−A×(10/92)+50  (44)

[0193] If 92≦A<184, then

S=−A×(10/46)+60  (45)

[0194] If 184≦A<230, then

S=−A×(10/23)+100  (46)

[0195] If 230≦A, then

S=0  (47)

[0196] In this manner, the saturation index ‘S’ is determined. Referringto FIG. 23, there is shown a relationship between minimum saturation ‘A’and saturation index ‘S’. As shown in this figure, the saturation index‘S’ increases in a range of maximum ‘50’ to minimum ‘0’ as thesaturation ‘A’ decreases, and it decreases as the saturation ‘A’increases. For conversion from the saturation index ‘S’ to thesaturation enhancement parameter ‘Sratio’, the following equation isapplicable:

Sratio=(S+100)/100  (48)

[0197] In this case, when the saturation index ‘S’ is ‘0’, thesaturation enhancement parameter ‘Sratio’ becomes ‘1’ not to allowsaturation enhancement.

[0198] The following describes a technique for enhancing chromaticitywhile specifying a range of pixels according to chromaticity as shown inFIG. 10. In principle, an approach for increasing luminance of pixels inchromaticity enhancement is employed. Therefore, a γ-correction tonecurve as shown in FIG. 20 is used. According to a degree of chromaticityenhancement, a value of γ can be adjusted, and it is also possible toset up a γ value automatically, which will be described later.

[0199] In the present preferred embodiment, there are provided suchimage processing techniques as mentioned hereinabove. Since acorrespondence relationship applicable to an object pixel is judged atstep S110 and applicability k is determined at steps S111 to S115, imagedata conversion is performed at step S120 according to result ofjudgment.

[0200] As mentioned above, on the processing menu area 43 shown in FIG.7, a kind of image processing is selected and a level of enhancement isspecified. Therefore, at step S100, an image processing conversion tablebased on respective enhance levels for image processing is prepared andstored into a predetermined memory area of the computer system. Then, atstep S120, the image processing conversion table is referenced to carryout conversion as described below.

[0201] Under condition that components of pre-converted RGB gradationdata are (Rpre, Gpre, Bpre), components of post-converted RGB gradationdata attained through reference to a predetermined conversion table are(Rpost, Gpost, Bpost), and components of final image data are (Rfinl,Gfinl, Bfinl), conversion is performed as expressed below.

Rfinl=k·Rpost+(1−k)·Rpre  (49)

Gfinl=k·Gpost+(1−k)·Gpre  (50)

Bfinl=k·Bpost+(1−k)·Bpre  (51)

[0202] In image processing based on these equations, a gradual increasein weight assignment is made in a transition region where applicabilityk varies from ‘0’ to ‘1’, resulting in elimination of a stepwisedifference.

[0203] Equations (49) to (51) are used in succession for all the imageprocessing conversion tables to which the object pixel is applicable. Ifapplicability k is ‘0’, it is not necessary to perform image dataconversion. As to Equations (49) to (51), each of red, green and bluecomponents is subjected to calculation, but just one or two of thesecomponents may be calculated in some cases. Still more, in such asituation where applicability k is not used, each component (Rpost,Gpost, Bpost) of RGB gradation data attainable just by changing a kindof conversion table may be used intactly.

[0204] In the foregoing description, an applicable object is specifiedfor each image processing, i.e., only a particular region in view of anentire image is subjected to image processing. It is however possible toperform additional image processing operation on a particular regionwhile carrying out a certain image processing operation in the entireimage.

[0205] Under condition that each component (Rtotal, Gtotal, Btotal) canbe attained as a result of image processing conversion for the entireimage and each component (Rpart, Gpart, Bpart) can be attained as aresult of image processing conversion for any particular region,weighted addition is performed using applicability k′ similar to theapplicability as shown below.

Rfinl=k′·Rpart+(1−k′)·Rtotal  (52)

Gfinl=k′·Gpart+(1−k′)·Gtotal  (53)

Bfinl=k′·Bpart+(1−k′)·Btotal  (54)

[0206] Referring to FIG. 24, there is shown a schematic example in whichan image of a slightly back-lighted person in a scene is selected toincrease its brightness while entire color vividness is enhanced. Aresult of conversion on a region of the entire image (Rtotal, Gtotal,Btotal), hatched by frontward-sloping diagonal lines, is laid over aresult of conversion on a region of the person's image (Rpart, Gpart,Bpart), hatched by backward-sloping diagonal lines, according toapplicability k′. In this case, on the region of the person's image,applicability k′ is set to a maximum level of 0.5, and on a transitionregion at a circumference thereof, applicability k′ is adjusted in arange of 0<k′<0.5. Still more, if applicability k′ is set to a maximumlevel of 1.0 on the region of the person's image and applicability k′ isadjusted in a range of 0<k′<1.0 on the transition region at thecircumference thereof, conversion for the entire image can be excludedfrom the region of the person's image. In the same manner, amultiplicity of image processing results can be applied.

[0207] Thereafter, at step S130, the object pixel is moved, and at stepS140, it is checked whether all the object pixels have been subjected toprocessing. If processing is completed for all the object pixels, theimage processing sequence concerned comes to an end.

[0208] Having described the present preferred embodiment as related totechniques for automatically setting up a degree of enhancement inrespective image processing operations, there may also be provided suchan arrangement that image data is sampled uniformly for statisticalcalculation at a step before movement of an object pixel as mentionedabove and a degree of enhancement is set up automatically according toresult of statistical calculation while a conversion table is created.

[0209] In an instance where a range of chromaticity is selected using anoptional function of the printer driver 21 b, an enhancement level canbe set up automatically in such a manner as mentioned above, therebymaking it possible to improve operability.

[0210] The following describes operations in the preferred embodimentarranged as stated above.

[0211] It is assumed that a photographic image is read in using thescanner 11 and printed out using the printer 31. First, under conditionthat the operating system 21 a is run on the computer 21, the imageprocessing application 21 d is launched to let the scanner 11 read inthe photographic image. When the photographic image thus read in istaken into the image processing application 21 d under control of theoperating system 21 a, the image processing application 21 d carries outimage processing operations as flowcharted in FIG. 5.

[0212] At step S100, the read-in photographic image is presented on thedisplay area 42 of the window 40 as shown in FIG. 7 so that anapplicable object can be specified. In this state, a human operatorspecifies a rectangular region of sky-blue part using the mouse 27 andclicks the up-arrow of the SATURATION adjustment item on the processingmenu area 43 as many times as desired for saturation enhancement. Stillmore, the operator specifies a rectangular region of person's image partat the center and clicks the up-arrow of the BRIGHTNESS adjustment itemon the processing menu area 43 as many times as desired for brightnessenhancement. Namely, while specifying an image processing operation ofsaturation enhancement on a rectangular region corresponding to a skypart of background, the operator specifies an image processing operationof brightness enhancement on a rectangular region corresponding to aperson's image part.

[0213] When the window 40 is closed on completion of specifications, aconversion table is created according to the specifications. That is, akind of image processing is determined with reference to akind-of-processing code indicated on the region reference table shown inFIG. 9, and a degree of enhancement is judged according to alevel-of-adjustment value indicated thereon for conversion tablecreation. In this example, a conversion table for saturation enhancementprocessing and a conversion table for brightness enhancement processingare created.

[0214] Thereafter, an object pixel applicable to processing is set at aninitial position, and at step S110, it is judged whether or not acoordinate location of the object pixel is included in the regionreference table shown in FIG. 9. In this case, a transition region isalso taken into account, and applicability k is attained. If the objectpixel is located in a hatched region as shown in FIG. 8, applicability kis set to ‘1’. Then, at step S120, with reference to a conversion tablefor applicable image processing, the image data is converted accordingto Equations (49) to (51). In an example shown in FIG. 8, therectangular region specifying the background part and the rectangularregion specifying the person's image part are overlaid partially, and anoverlaid part is subjected to image data conversion through two stepsusing Equations (49) to (51). In a transition region, image dataconversion is also performed to eliminate a stepwise difference along anon-specified circumferential part.

[0215] The above-mentioned operations are repeated while moving eachobject pixel applicable to processing at step S130 until it is judge atstep S140 that all the pixels have been processed. Thus, saturation isenhanced on the sky-blue part to provide a clear blue sky image, whilethe person's part is brightened to provide such a blight image asphotographed with flash light even if the person's image is back-lightedin scene. It is to be understood that the blue-sky part is notbrightened due to enhancement of brightness of the person's part andsaturation of the person's part is not increased due to enhancement ofsaturation of the blue-sky part.

[0216] Thereafter, image data thus processed is displayed on the displaymonitor 32 through the display driver 21 c, and if the image thusdisplayed is satisfactory, it is printed out on the printer 31 throughthe printer driver 21 b. More specifically, the printer driver 21 breceives RGB gradation image data which has been subjected to imageprocessing as specified for each specified region, performs resolutionconversion as predetermined, and carries out rasterization according toa print head region of the printer 31. Then, the image data thusrasterized is subjected to RGB-to-CMYK color conversion, and thereafter,CMYK gradation image data is converted into binary image data for outputonto the printer 31.

[0217] In an instance where the printer driver 21 b is made active by arequest for print processing from a certain application unlike theabove-mentioned image processing through the image processingapplication 21 d, an image to be read in may not be displayed on thedisplay area 42 of the window 40. In this case, the printer driver 21 bcan present such an option selection window as shown in FIG. 10. On thiswindow, the operator specifies a desired image processing item to attaina clear flesh color part of a person's image or a vivid green part oftree leaves, for example, while observing an original photograph or thelike. This processing operation corresponds to step S100 in FIG. 5 atwhich an applicable object is specified.

[0218] After an optional processing item is selected, the printer driver21 b creates a conversion table internally and judges chromaticity ofeach pixel of input image data. Thus, it is checked whether chromaticityof each pixel is applicable to the selected optional processing item. Ifit is applicable, image processing conversion is carried out usingEquations (49) to (51) with reference to the conversion table, and imagedata thus converted is further converted into CMYK image data which canbe printed out onto the printer 31.

[0219] In such a manner as mentioned above, flesh color pixels of aperson's image and green color pixels of tree images in an originalscene are enhanced through adjustment, resulting in vivid imaging onprintout.

[0220] Thus, on the computer 21 serving as the nucleus of imageprocessing, a region applicable to image processing is specified at stepS100, an object pixel is moved for judging whether or not it belongs tothe specified region at steps S110 to S140, and then a specified imageprocessing operation is carried out if the object pixel is judged tobelong to the specified region. Therefore, adjustment of image data in acertain region does not have an adverse effect on image data in anotherregion, making it possible to realize satisfactory adjustment in anentire image with ease.

[0221] While chromaticity enhancement is made in a range of pixelsspecified according to chromaticity in the example mentioned above, itis also practicable to set up a degree of enhancement automatically. Thefollowing describes details of a color adjustment device implemented forthe purpose of automatic setting of a degree of enhancement.

[0222] Referring to FIG. 25, there is shown a block diagram of a coloradjustment device in a preferred embodiment of the present invention. Aconcrete example of a hardware configuration of this system is similarto that illustrated in FIG. 2.

[0223] In FIG. 25, an image input device 70 supplies photographic colorimage data containing dot-matrix pixels to a color adjustment device 80,on which optimum color adjustment processing for the color image data iscarried out. Then, the color adjustment device 80 deliverscolor-adjusted image data to an image output device 90, on which acolor-adjusted image is output in a dot-matrix pixel form. In thissequence, the color-adjusted image data delivered by the coloradjustment device 80 is produced through judging an object and color ofimage according to chromaticity of each pixel and determining aprinciple and degree of optimum color adjustment in terms of colorcorrection of an entire image. For this purpose, the color adjustmentdevice 80 comprises the unit for judging chromaticity, the unit forstatistical calculation of object chromaticity pixels, the unit forjudging a degree of color adjustment, and the unit for adjusting color.

[0224] To be more specific, judgment on an object and color adjustmentthereof are carried out by a color adjustment processing program asflowcharted in FIG. 26, which is run on the computer 21. Note that aflowchart exemplified in FIG. 26 is for color adjustment processing toprovide clear flesh color.

[0225] In the color adjustment processing, statistical calculation isperformed on flesh-color-like pixels according to chromaticity of eachpixel. As shown in FIG. 6, each object pixel is moved for statisticalcalculation on all the pixels.

[0226] First, at step S210, chromaticity “x-y” of each pixel iscalculated. As in the case of the example in the foregoing description,flesh color is identified if the following expressions are satisfied.

0.35<x<0.40  (5)

0.33<y<0.36  (6)

[0227] At step S220, it is judged whether or not chromaticity “x-y”converted according to each pixel of RGB gradation data is in apredefined flesh color range. If it is in the flesh color range,statistical calculation is performed on each pixel of color image dataat step S230. This statistical calculation signifies simple addition ofRGB gradation data values. The number of pixels is also counted todetermine an average value for pixels judged to have flesh color, whichwill be described in detail later.

[0228] Thereafter, regardless of whether or not each object pixel isjudged to have flesh color, each object pixel is moved at step S240.Thus, the above-mentioned sequence is repeated until it is judged atstep S250 that processing for all the pixels is completed. On completionof processing for all the pixels, step S260 is performed to dividestatistical result data by the number of pixels for determining anaverage value (Rs.ave, Gs.ave, Bs.ave).

[0229] Software processing for “x-y” chromaticity calculation at stepS210 and hardware for execution thereof provide the unit for judgingchromaticity. It is judged at step S220 whether or not chromaticity“x-y” is in a predetermined object range, and if the predeterminedobject range is satisfied, statistical calculation is performed on colorimage data at step S230. Then, at steps S240 and S250, each object pixelis moved until all the pixels are taken. At step S260, statisticalresult is divided by the number of pixels to determine an average value.These software processing operations and hardware for execution thereofprovide the unit for statistical calculation of object chromaticitypixels.

[0230] As to pixels having preferable flesh color, an ideal value(Rs.ideal, Gs.ideal, Bs.ideal) is predefined. In terms of memory colorin psychology, each ideal value is different from result of actualmeasurement. In an example of flesh color, a person tends to have anoptical illusion that slightly deviated flesh color is true rather thanflesh color conforming to actual measurement result. This opticalillusion is based on a stereotyped recognition of flesh color onphotographs and pictures, i.e., it is referred to as a memory coloreffect in psychology. In the present invention, an ideal value ispredefined in consideration of such a memory color effect so that coloradjustment is made to eliminate a difference from the ideal value.Therefore, the ideal value may be in a wide target range expectedwithout being biased to actual color.

[0231] Regarding flesh color pixels, a difference between an averagevalue (Rs.ave, Gs.ave, Bs.ave) of RGB gradation data and an ideal value(Rs.ideal, Gs.ideal, Bs.ideal) predefined for preferable flesh colorrepresents a degree of deviation in color image data fundamentally.

[0232] However, it is not preferred to apply the difference as a degreeof color adjustment intactly. For instance, if the same degree of coloradjustment is applied to all the pixels, a flesh color part may becomesatisfactory but colors of pixels on any parts other than the fleshcolor part may be affected significantly.

[0233] In the present preferred embodiment, therefore, a ratio of thenumber of flesh color pixels to the total number of pixels (flesh colorratio) is determined at step S270 for regulating a degree of coloradjustment. Degrees of adjustment of primary colors ΔR, ΔG and ΔB areexpressed as shown below.

ΔR=ks(Rs.ideal−Rs.ave)  (55)

ΔG=ks(Gs.ideal−Gs.ave)  (56)

ΔB=ks(Bs.ideal−Bs.ave)  (57)

[0234] Based on these equations, a value of flesh color ratio ‘ks’ isattained as indicated below.

ks=(Number of flesh color pixels/Total number of pixels)

[0235] A degree of color adjustment thus attained is not appliedintactly to color image data adjustment. In the present preferredembodiment, a tone curve is prepared using the degree of coloradjustment at step S280. FIG. 27 is a diagrammatic illustration showingtone curves prepared in the present embodiment.

[0236] A tone curve represents an input-output relationship where RGBgradation data is converted with a degree of enhancement regulated. Inan example of 256 gradations ranging from levels ‘0’ to ‘255’, a splinecurve is drawn with respect to three identified output value pointscorresponding to gradation level ‘0’, gradation level ‘255’ and acertain medium gradation level therebetween. Assuming that mediumgradation level ‘64’ is taken and output values are ‘0’, ‘64’ and ‘255’,there is a coincidence in an input-output relationship even if inputvalues are ‘0’, ‘64’ and ‘255’, resulting in a tone curve beingstraightened. However, if output value ‘64’ is not provided for inputvalue ‘64’, a gentle curve as shown in FIG. 27 is drawn to set up aninput-output relationship. In the present preferred embodiment, acontrol point corresponding to the average value (Rs.ave, Gs.ave,Bs.ave) of RGB gradation data is used as a medium gradation level, andrespective degrees of color adjustment ΔR, ΔG and ΔB are reflected information of a tone curve. In this fashion, the control point is changedso that each ideal value (Rs.ideal, Gs.ideal, Bs.ideal) is met when theflesh color ratio ‘ks’ is ‘1’.

[0237] At step S290, element colors of color image data are convertedfor all the pixels again using a tone curve thus attained to accomplishcolor adjustment of the color image data.

[0238] Software processing at step S270 where color adjustment is madeaccording to a ratio of the number of object pixels to the total numberof pixels while determining a difference between a statistical resultvalue and an ideal value, software processing at step S280 where a tonecurve is formed according to a determined degree of color adjustment,and hardware for execution thereof provide the unit for judging a degreeof color adjustment. Software processing at step S290 where color imagedata is converted and hardware for execution thereof provide the unitfor adjusting color.

[0239] Having described the present preferred embodiment as related toflesh color adjustment for the purpose of simplicity in explanation, itis to be understood that color adjustment is not limited to flesh color.In consideration of a memory color effect in psychology, it is oftendesired to attain more vivid green color of tree leaves and clearer bluecolor of sky in addition to clearer flesh color through color adjustmentprocessing. Referring to FIG. 28, there is shown a modified embodimentin which an object of adjustment is selectable.

[0240] In the example shown in FIG. 28, an object of color adjustment isselected first at step S 305. In use of the computer 21, a window shownin FIG. 29 is presented on the display monitor 32 so that a humanoperator can select an object of color adjustment. The windowexemplified in FIG. 29 is provided with a flesh color adjustment itemfor clearer flesh color, a green color adjustment item for more vividgreen color of tree leaves, and a blue color adjustment item for clearersky blue, each of which has a check box for allowing individualselection. In this example, duplicate selection is also permitted. Whenthe operator turns on a desired check box and click the ‘OK’ button, aflag for each object thus specified is set up to start a loop processingof steps S310 to S350.

[0241] In the loop processing, while moving an object pixel, statisticalcalculation is performed on each pixel through determining chromaticityas in the foregoing description. At step S310, object pixel chromaticity“x-y” is determined using Equations (1) to (4). Then, at step S315, aflesh color adjustment flag which has been set up at step S305 isreferenced to judge whether or not the operator has selected the fleshcolor adjustment item. If it has been selected, statistical processingfor flesh color pixels is performed. This statistical processing iscarried out in the same manner as at steps S220 and S230 in the previousexample. If chromaticity “x-y” determined at step S310 is in apredefined possible chromaticity range corresponding to flesh color,statistical calculation is performed for each element color of RGBgradation data.

[0242] At step S325, a green color adjustment flag is referenced tojudged whether or not the operator has selected the green coloradjustment item in the same manner as for flesh color adjustment. If thegreen color adjustment item has been selected, object pixel chromaticity“x-y” is checked to judge whether or not it is in a predefined possiblechromaticity range corresponding to green color of tree leaves. If it isin the corresponding predefined chromaticity range, statisticalcalculation is performed at step S330. This statistical calculation iscarried out in an area different from that subjected to flesh colorstatistical calculation.

[0243] Then, at step S335, the blue color adjustment item is checked toform a judgment in the same manner, and at step S340, statisticalcalculation is performed on another area.

[0244] At step S345, each object pixel is moved, and the above-mentionedsequence is repeated until it is judged at step S350 that processing forall the pixels is completed. In this modified embodiment, a plurality ofobjects may be selected for color adjustment. Even in such a situation,statistical calculation is performed at steps S315 to S340 ifchromaticity of each object pixel is in a predefined chromaticity range.Therefore, these processing operations provide the unit for statisticalcalculation of object chromaticity pixels.

[0245] At steps S355 to S365 after completion of chromaticitystatistical calculation on all the pixels, a degree of adjustment foreach color is calculated according to result of statistical calculation.Unlike the previous example, a processing operation for determining anaverage value from statistical calculation result is performedsimultaneously with calculation of a degree of each color adjustment inthis modified embodiment, and it is possible to modify relevantcalculation procedures as required. As described in the previousexample, each adjustment of flesh color, green color and blue color iscarried out in the same manner. That is, an average value is calculatedaccording to statistical calculation result, a difference between it andan ideal value predefined for preferable color is determined, andmultiplication by a flesh color ratio, green color ratio or blue colorratio is performed for regulating a degree of each color adjustment.

[0246] Upon completion of step S365, there are provided three kinds ofdegrees of color adjustment since degrees of flesh color adjustment,green color adjustment and blue color adjustment have been determinedthrough respective statistical calculations. Therefore, in the modifiedembodiment, addition is performed to reflect results of thesestatistical calculations inclusively. More specifically, in a situationof wider application of processing objects, degrees of color adjustmentfor respective processing objects ΔR, ΔG and ΔB are determined asexpressed below. $\begin{matrix}{{\Delta \quad R} = {\sum\limits_{i}{{ki}\quad \left( {{{Ri} \cdot {ideal}} - {{Ri} \cdot {ave}}} \right)}}} & (58) \\{{\Delta \quad G} = {\sum\limits_{i}{{ki}\left( {{{Gi} \cdot {ideal}} - {{Gi} \cdot {ave}}} \right)}}} & (59) \\{{\Delta \quad B} = {\sum\limits_{i}{{ki}\left( {{{Bi} \cdot {ideal}} - {{Bi} \cdot {ave}}} \right)}}} & (60)\end{matrix}$

[0247] where $\begin{matrix}{{\sum\limits_{i}{ki}}<=1} & (61)\end{matrix}$

[0248] i=1: flesh color

[0249] i=2: green color

[0250] i=3: blue color

[0251] In this case, it is assumed that no duplicate counting is made.

[0252] At step S370, a tone curve is formed according to each of degreesof color adjustment ΔR, ΔG and ΔG thus determined, as shown in FIG. 27.In this case, control points are indicated by Σ ki·Ri.ave, Σ ki·Gi.ave,Σ ki·Bi.ave. Software processing operations at steps S355 to S370provide the unit for judging degree of color adjustment. After formationof each tone curve, color image data is adjusted at step S375.

[0253] The above-mentioned color adjustment device may also beimplemented as a printer driver. Inmost cases, a printer driver is notcapable of temporarily storing data in an output process afterprocessing of input data. Hence, there is a certain limitation infunctionality for changing processing conditions according to eachregion divided as desired. However, by setting up degrees of coloradjustment for a plurality of elements as shown in Equations (58) to(60), it is possible to carry out effective color adjustment even forthe printer driver having such a functional limitation.

[0254] The following describes operations of a preferred embodimentarranged with a printer driver.

[0255] As in the previous exemplary embodiment, it is assumed that aphotographic color image shown in FIG. 30 is read in using the scanner11 and printed out using the printer 31. First, under condition that theoperating system 21 a is run on the computer 21, the color adjustmentapplication 21 d is launched to let the scanner 11 read in thephotographic image. When the photographic image thus read in is takeninto the color adjustment application 21 d under control of theoperating system 21 a, an object pixel applicable to processing is setat an initial position. Then, at step S210, chromaticity “x-y” of eachpixel is calculated using Equations (1) to (4). At step S220, it isjudged whether or not each of values ‘x’ and ‘y’ is in a predefinedflesh color chromaticity range. If it is in the flesh color chromaticityrange, statistical calculation is performed on each pixel of color imagedata for each element color at step S230. In the photographic imageshown in FIG. 30, pixels of person's hands, legs or face can be judgedto have flesh color. In this example, statistical calculation isperformed on a few percent of all the pixels as flesh color pixels.Then, at step S240, each object pixel is moved. Thus, theabove-mentioned sequence is repeated until it is judged at step S250that processing for all the pixels is completed. After completion ofprocessing for all the pixels, statistical result data is divided by thenumber of flesh color pixels to determine an average value at step S260.At step S270, a difference between an ideal value of flesh color and theaverage value of flesh color pixels is determined, and it is multipliedby a flesh color ratio that represents a ratio of the number of fleshcolor pixels to the total number of pixels. At step S280, a tone curveis formed accordingly. Then, at step S290, based on the tone curve, eachelement color of color image data is converted for color adjustment.Since a degree of color adjustment is set at a moderate level withrespect to the ideal value of flesh color in consideration of the ratioof the number of flesh color pixels to the total number of pixels,preferable color can be attained through proper color adjustment.

[0256] In the example shown in FIG. 30, a difference between an averagevalue of flesh color pixels attained in statistical calculation and anideal value of flesh color is multiplied by a flesh color ratioindicating a few percent value for regulating a degree of coloradjustment. According to the regulated degree of color adjustment, atone curve is formed for accomplishing color adjustment.

[0257] Still more, if a flesh color adjustment item, green coloradjustment item and blue color adjustment item are selected inadjustment object selection as exemplified before, chromaticity “x-y” iscalculated for all the pixels at step S310. Then, at steps S315 to S340,individual statistical calculation is performed for each adjustmentobject. In the example shown in FIG. 30, on a flesh color part of aperson's image, a green color part of tree leaves and a sky-blue part ofa background, each chromaticity “x-y” is applicable to each object rangefor statistical calculation.

[0258] After completion of processing for all the pixels, a degree ofcolor adjustment for each object is determined in consideration of anoccupancy ratio of object pixels at steps S355 to S365. At step S370, atone curve is formed through regulating each degree of color adjustmentthus determined. Then, at step S375, color adjustment is carried out forall the pixels of color image data. In this manner, flesh coloradjustment for attaining clearer flesh color, green color adjustment forattaining more vivid green color of tree leaves, and blue coloradjustment for attaining clearer sky blue in background are carried outthrough regulation according to the occupancy ratio of object pixels ofeach color.

[0259] After color adjustment thus accomplished, a color image isdisplayed on the display monitor 32 through the display driver 21 c, andthen if the color image thus displayed is satisfactory, it is printedout onto the printer 31 through the printer driver 21 b. Morespecifically, the printer driver 21 b receives RGB gradation image datawhich has been subjected to color adjustment, performs resolutionconversion as predetermined, and carries out rasterization according toa print head region of the printer 31. Then, the image data thusrasterized is subjected to RGB-to-CMYK color conversion, and thereafter,CMYK gradation image data is converted into binary image data for outputonto the printer 31.

[0260] Through the above-mentioned processing, the photographic colorimage read in using the scanner 11 is automatically subjected to optimumcolor adjustment. Thereafter, it is displayed on the display monitor 32,and then printed out onto the printer 31.

[0261] As set forth hereinabove, on the computer serving as the nucleusof color adjustment, chromaticity “x-y” of each pixel is calculated atstep S210, and statistical calculation is performed at steps S220 toS230 if a value of chromaticity thus calculated is in a chromaticityrange predefined for each color. After completion of statisticalcalculation on all the pixels, an average value is determined at stepS260, and a degree of each color adjustment is calculated while takingaccount of an occupancy ratio of object pixels of each color at stepS270. In this fashion, accurate statistical calculation is performed oncolor pixels to be adjusted independently of brightness, and a degree ofeach color adjustment is regulated by taking account of the number ofpixels of each color in terms of occupancy ratio, thereby making itpossible to carry out optimum color adjustment processing without givingan adverse effect on colors of pixels surrounding object pixels.

[0262] The invention may be embodied in other specific forms withoutdeparting from the spirit or essential characteristics thereof. Thepresent embodiments are therefore to be considered in all respects asillustrative and not restrictive, the scope of the invention beingindicated by the appended claims rather by the foregoing description andall changes which come within the meaning and range of equivalency ofthe claims are therefore intended to be embraced therein.

What is claimed is:
 1. An image processing apparatus for inputtingphotographic image data containing dot-matrix pixels and converting eachpixel of said image data according to predetermined correspondencerelationship information, comprising: correspondence relationshipholding the unit which holds a plurality of correspondence relationshipinformation for conversion of said image data in combination withapplicable object information thereof; correspondence relationshipjudging unit which judges each correspondence relationship forconverting each pixel of said image data according to saidcorrespondence relationship and applicable object information which isheld by said correspondence relationship holding unit; and image dataconverting unit which converts each pixel of said image data byreferencing each correspondence relationship judged by saidcorrespondence relationship judging unit from said correspondencerelationship holding unit.
 2. An image processing apparatus as claimedin claim 1, wherein, in said correspondence relationship informationholding unit, each correspondence relationship applicable to each partof image being held along with information on each part applicable toeach correspondence relationship, and wherein, in the step of forming ajudgment on said correspondence relationship, a part of imagecorresponding to each pixel being detected and each correspondencerelationship for conversion being judged through comparing the detectedpart of image with part information for each correspondencerelationship.
 3. An image processing apparatus as claimed in claim 1,wherein, in said correspondence relationship information holding unit, aplurality of correspondence relationships being held along withinformation on chromaticity applicable to each correspondencerelationship, and wherein, in said correspondence relationship judgementunit, a value of chromaticity of each pixel being detected and eachcorrespondence relationship for conversion being judged throughcomparing the detected value of chromaticity with information onchromaticity for each correspondence relationship.
 4. An imageprocessing apparatus as claimed in claim 1, wherein, in saidcorrespondence relationship information holding unit, being provided acolor conversion table which stores information on correspondencerelationship between pre-converted original image data,andpost-converted image data.
 5. An image processing apparatus as claimedin claim 1, wherein, in said correspondence relationship informationholding unit, a plurality of correspondence relationships being realizedby changing a degree of correspondence relationship applicability.
 6. Animage processing apparatus as claimed in claim 1, wherein, in saidcorrespondence relationship information holding unit, a plurality ofcorrespondence relationships applicable to said image data beingspecified in succession as required.
 7. An image processing apparatus asclaimed in claim 1, wherein, in said correspondence relationshipinformation holding unit, each correspondence relationship for adjustingbrightness according to said image data being held.
 8. An imageprocessing apparatus as claimed in claim 1, wherein, in saidcorrespondence relationship information holding unit, eachcorrespondence relationship for adjusting chromaticity according to saidimage data being held.
 9. An image processing apparatus as claimed inclaim 1, wherein, in said correspondence relationship informationholding unit, each correspondence relationship for adjusting colorvividness according to said image data being held.
 10. An imageprocessing apparatus as claimed in claim 1, wherein, in said image dataconverting unit, for pixels in a transition region between regions whichbeing different in terms of correspondence relationship, a level ofcorrespondence relationship is adjusted gradually in image dataconversion.
 11. An image processing method of inputting photographicimage data containing dot-matrix pixels and converting each pixel ofsaid image data according to predetermined correspondence relationshipinformation, comprising the steps of: holding a plurality ofcorrespondence relationship information for conversion of said imagedata in combination with applicable objet information thereof; forming ajudgment on each correspondence relationship for converting each pixelof said image data according to said correspondence relationship andapplicable object information thus held; and converting each pixel ofsaid image data according to result of judgment thus formed.
 12. Astorage medium for recording an image processing program for inputtingphotographic image data containing dot-matrix pixels on a computer andconverting each pixel of said image data according to predeterminedcorrespondence relationship information, said image processing programcomprising the steps of: holding a plurality of correspondencerelationship information for conversion of said image data incombination with applicable object information thereof; forming ajudgment on each correspondence relationship for converting each pixelof said image data according to said correspondence relationship andapplicable object information thus held; and converting each pixel ofsaid image data according to result of judgment thus formed.
 13. A coloradjustment device for performing color separation of color image datafor each predetermined element color and adjusting said color image datain enhancement to provide each desired color in result of color imageoutput delivered for each element color by such a device as an imageoutput device, etc., comprising: chromaticity judging unit whichdetermines a value of chromaticity of each pixel according to said colorimage data; object chromaticity pixel statistical calculation unit whichperforms statistical calculation on pixels having chromaticity valueswhich have been determined to meet a predefined range of chromaticity bysaid chromaticity judging unit; color adjustment degree judging unitwhich determines a degree of color adjustment so as to eliminate adifference between a predetermined optimum value for pixels meeting saidpredefined range of chromaticity and a result value attained in saidstatistical calculation and for regulating said degree of coloradjustment according to an occupancy ratio of pixels subjected to saidstatistical calculation to the total number of pixels; and coloradjusting unit which carries out color adjustment of said image dataaccording to the regulated degree of color adjustment.
 14. A coloradjustment device as claimed in claim 13, wherein: in said objectchromaticity pixel statistical calculating unit, an average; value ofpixels judged to be object pixels being calculated for each elementcolor of color image data; in said color adjustment degree judging unit,an optimum value of each element color being provided for color imagedata meeting said predefined range of chromaticity, and a degree ofcolor adjustment being determined according to said optimum value ofeach element color.
 15. A color adjustment device as claimed in claim13, wherein, in said chromaticity pixel statistical calculating unit,statistical calculation being performed on object chromaticity pixels ina possible range of chromaticity in terms of memory color in psychology.16. A color adjustment device as claimed in claim 13, wherein, in acolor adjustment degree judging unit, a tone curve representinginput-output relationship information being used for changing a degreeof element color enhancement, and a tone curve being formed according tosaid degree of color adjustment.
 17. A color adjustment method ofperforming color separation of color image data for each predeterminedelement color and adjusting said color image data in enhancement toprovide each desired color in result of color image output delivered foreach element color by such a device as an image output device, etc.,comprising the steps of: determining a value of chromaticity of eachpixel according to said color image data; performing statisticalcalculation on pixels having chromaticity values which have beendetermined to meet a predefined range of chromaticity; determining adegree of color adjustment so as to eliminate a difference between apredetermined optimum value for pixels meeting said predefined range ofchromaticity and a result value attained in said statisticalcalculation; regulating said degree of color adjustment according to anoccupancy ratio of pixels subjected to said statistical calculation tothe total number of pixels; and carrying out color adjustment of saidimage data according to the regulated degree of color adjustment.
 18. Astorage medium for recording a color adjustment control program forperforming color separation of color image data for each predeterminedelement color on a computer and adjusting said color image data inenhancement to provide each desired color in result of color imageoutput delivered for each element color by such a device as an imageoutput device, etc., said color adjustment control program comprisingthe steps of: determining a value of chromaticity of each pixelaccording to said color image data, performing statistical calculationon pixels having chromaticity values which have been determined to meeta predefined range of chromaticity, determining a degree of coloradjustment so as to eliminate a difference between a predeterminedoptimum value for pixels meeting said predefined range of chromaticityand a result value attained in said statistical calculation; regulatingsaid degree of color adjustment according to an occupancy ratio ofpixels subjected to said statistical calculation to the total number ofpixels; and carrying out color adjustment of said image data accordingto the regulated degree of color adjustment.